fix(ci): route every invalid_payload verdict through a shared builder by groupthinking · Pull Request #1357 · groupthinking/EventRelay

groupthinking · 2026-08-04T05:29:04Z

Canonical issue

Closes #

Outcome

The agent completion gate (scripts/ci/agent_completion_gate.py) now surfaces the collector's own collection_errors on every invalid_payload verdict, not only on the late field-validation path. When a payload is rejected early (non-dict payload, missing/invalid policy, non-bool policy.applicable), the author now sees the collector diagnostics that explain why, instead of an empty details object. The gate stays fail-closed — verdict and reasons are byte-identical for every input; only details is enriched.

Scope

Included:
- New _invalid_payload(payload, invalid_fields) helper that builds a single, consistent invalid_payload verdict and folds in surfaced collection_errors.
- New _collection_errors(payload) helper (extracted from the inline comprehension) so both the happy path and every invalid_payload path share one implementation.
- All four early invalid_payload return sites route through the shared builder.
- Regression tests asserting each early-reject path preserves verdict/reasons and surfaces collection errors when present.
Explicitly excluded: no change to gate policy semantics, thresholds, or the not_applicable / evidence_collection_failed paths.

Risk

Risk level: low
Failure mode: a bug here could only widen details; verdict/reasons are covered by tests asserting byte-identical output, so a regression cannot flip the gate open or closed.
Rollback: revert the two commits; the gate returns to emitting empty details on early rejects.

Verification

Tied to head SHA 5a034b77182e04c227f09b18a481fe364233acec.

Focused tests — PYTHONPATH=src pytest tests/unit/test_agent_completion_gate.py → 109 passed, 89 subtests passed
Required CI — pending (draft)
Review threads resolved — none open

Note: scripts/ is outside the CI ruff scope (ci.yml lints only src/youtube_extension/backend/ and main.py); the new helpers follow the file's existing Dict/List typing convention.

Production evidence

Not applicable — this is a CI-gate script change with no runtime or deployment surface.

Agent handoff

One canonical issue is linked
No competing PR implements the same issue
Acceptance criteria are satisfied
Required checks pass on the current head
Human decision is requested only for product, security, irreversible infrastructure, or production approval

Human sign-off is requested at the merge gate: base branch main is protected, so this draft is left for a human to mark ready and merge.

Generated by Claude Code

…verdicts The agent-completion truth gate blocks ~47 of the 69 open PRs with a bare `invalid_payload` and no remediation path. Root cause: `agentTaskApplicable()` in pr-checks.yml classifies any branch matching /^(?:agent|claude|codex|copilot|jules)[\/-]/ as agent work, so human-authored Claude Code worktree branches are held to the full AgentTask provenance contract. With no linked AgentTask issue, the collector emits `policy.agent_login` and `policy.run_id` as null and records the real reasons in `collection_errors` (missing_linked_issue, missing_agent_login, missing_agent_run_id). `evaluate()` then returned at the schema check and discarded `collection_errors` entirely -- they are only read further down, after the early return. Authors saw `invalid_payload` and nothing else. This keeps the gate fail-closed and byte-identical in `verdict` and `reasons`, and only adds `details.collection_errors` so the gate says what to fix. Verified: 112 passed against the reproduced PR #1270 payload. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

The collection_errors diagnostic was only attached to the late field-validation return. The three early invalid_payload short-circuits (payload not a dict, policy not a dict, missing/invalid policy.applicable) returned bare verdicts, so a malformed payload that never reaches field validation stayed just as opaque despite the collector having already recorded why — exactly the case the review thread raised (evaluate({"policy": {}, "collection_errors": [...]})). Extract _invalid_payload(payload, invalid_fields) and route all four invalid_payload returns through it so the diagnostic is applied consistently. verdict and reasons stay byte-identical for every input; only details is enriched, and only when the collector recorded errors — the gate remains fail-closed. Add regression coverage for both early paths and confirm a non-dict payload still returns an empty details. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011L6jqdhrKTYLinTnJYKEg9

vercel · 2026-08-04T05:29:10Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
v0-uvai	Ready	Preview, v0	Aug 4, 2026 5:29am

coderabbitai · 2026-08-04T05:29:13Z

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)

[‘architecture-gap’, ‘bug’, ‘ci-cd’, ‘ci/cd’, ‘copilot-rabbit’, ‘documentation’, ‘duplicate’, ‘enhancement’, ‘frontend’, ‘github_actions’, ‘good first issue’, ‘help wanted’, ‘high-priority’, ‘invalid’, ‘javascript’, ‘ml-model’, ‘needs-triage’, ‘pipeline-critical’, ‘placeholder-code’, ‘priority:high’, ‘python’, ‘python:uv’, ‘question’, ‘styling’, ‘tests’, ‘v0’]

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Repository UI (inherited), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 951d36bd-9bf4-4d79-b8f4-e134e69e0da6

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-08-04T05:29:18Z

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Snapshot Warnings

⚠️: No snapshots were found for the head SHA 5a034b7.

Ensure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice.

Scanned Files

None

github-actions · 2026-08-04T05:29:35Z

Agent Completion Truth Gate: BLOCKED

Reasons: invalid_payload

Machine-readable verdict

{
  "details": {
    "collection_errors": [
      "incomplete_linked_issue_contract",
      "missing_linked_issue",
      "missing_closing_issue_reference",
      "missing_agent_run_id",
      "missing_agent_login"
    ],
    "invalid_fields": [
      "issue.number",
      "policy.agent_login",
      "policy.run_id"
    ]
  },
  "reasons": [
    "invalid_payload"
  ],
  "verdict": "blocked"
}

Workflow evidence

groupthinking · 2026-08-04T05:34:02Z

Closing as a duplicate of the canonical open PR #1331 (Closes #1286, "Lands #1285"), which implements the identical change (+156/−26 across scripts/ci/agent_completion_gate.py and tests/unit/test_agent_completion_gate.py) on a land-pr-* branch already rebased onto current main.

Why this PR is redundant and being withdrawn rather than driven to green:

Same issue, established PR already open. fix(ci): surface collection errors behind invalid_payload truth-gate verdicts #1331 predates this PR and is the non-draft vehicle for Truth gate reports bare invalid_payload and discards the collector's diagnostics, stranding ~47 open PRs #1286. Opening a second PR for the same issue violates the "no competing PR implements the same issue" rule that fix(ci): surface collection errors behind invalid_payload truth-gate verdicts #1331 itself asserts. This one was opened before I discovered fix(ci): surface collection errors behind invalid_payload truth-gate verdicts #1331.
The agent-completion/truth-gate red here is the known Truth gate reports bare invalid_payload and discards the collector's diagnostics, stranding ~47 open PRs #1286 false-positive, not a defect. This branch is claude/*, so agentTaskApplicable() classifies it as agent work and demands a provenance contract no linked issue declares — the exact over-triggering documented in Truth gate reports bare invalid_payload and discards the collector's diagnostics, stranding ~47 open PRs #1286 (out of scope for the fix) and agent-completion/truth-gate is permanently unsatisfiable for any PR closing an [aw] issue #1130. fix(ci): surface collection errors behind invalid_payload truth-gate verdicts #1331 lives on a non-claude/* branch and so is not tripped by it. The verdict cited only metadata fields (issue.number, policy.agent_login, policy.run_id), never the collection_errors path this change touches, confirming it is not a regression from the diff.
No honest way to green it here. Fabricating an Agent Run ID / Agent Login manifest to satisfy the gate would defeat exactly what the gate protects (see agent-completion/truth-gate is permanently unsatisfiable for any PR closing an [aw] issue #1130) — and it would be self-defeating on a PR whose own purpose is to strengthen that gate.

Focused tests pass locally on these commits (pytest tests/unit/test_agent_completion_gate.py → 109 passed, 89 subtests). Issue #1286 is fully implemented; the remaining step is a human merge decision on #1331 into protected main. No further code work is needed.

Generated by Claude Code

groupthinking · 2026-08-04T06:00:07Z

Re-closing — this PR was reopened by an external actor (not a request I received), but nothing has changed that alters the earlier determination: it's a duplicate of the canonical open PR #1331 (Closes #1286), which carries the identical +156/−26 change on a land-pr-* branch that isn't tripped by the truth-gate false-positive, and which I've now reviewed (LGTM, 114 focused tests passing).

On the agent-completion/truth-gate red here: note it now surfaces collection_errors (missing_linked_issue, missing_agent_run_id, missing_agent_login, …) alongside the bare invalid_payload — that's the #1286 fix itself working, giving the diagnostics that were previously discarded. It still blocks because this is a claude/* branch with no linked AgentTask, the known over-triggering false-positive (out of scope for the fix, tracked in #1286). There is no honest way to green it: fabricating an Agent Run ID / Agent Login manifest would defeat exactly what the gate protects, and doubly so on a change whose purpose is to strengthen that gate.

Deferring to #1331 for the merge. If you intentionally reopened this and want it kept open in preference to #1331, reopen and say so and I'll follow that instead.

Generated by Claude Code

groupthinking and others added 3 commits August 4, 2026 03:02

Merge origin/main into claude/determined-maxwell-vunivy

5a034b7

github-actions Bot added the python label Aug 4, 2026

vercel Bot deployed to Preview August 4, 2026 05:29 View deployment

groupthinking closed this Aug 4, 2026

groupthinking reopened this Aug 4, 2026

groupthinking closed this Aug 4, 2026

Navigation Menu

fix(ci): route every invalid_payload verdict through a shared builder - #1357

fix(ci): route every invalid_payload verdict through a shared builder#1357
groupthinking wants to merge 3 commits into
mainfrom
claude/determined-maxwell-vunivy

groupthinking commented Aug 4, 2026

Uh oh!

vercel Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 4, 2026 •

edited

Loading

Review skipped

Uh oh!

github-actions Bot commented Aug 4, 2026

Uh oh!

github-actions Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

groupthinking commented Aug 4, 2026

Uh oh!

groupthinking commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

groupthinking commented Aug 4, 2026

Canonical issue

Outcome

Scope

Risk

Verification

Production evidence

Agent handoff

Uh oh!

vercel Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

github-actions Bot commented Aug 4, 2026

Dependency Review

Snapshot Warnings

Scanned Files

Uh oh!

github-actions Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Agent Completion Truth Gate: BLOCKED

Uh oh!

groupthinking commented Aug 4, 2026

Uh oh!

groupthinking commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel Bot commented Aug 4, 2026 •

edited

Loading

coderabbitai Bot commented Aug 4, 2026 •

edited

Loading

github-actions Bot commented Aug 4, 2026 •

edited

Loading